The developmental analysis of demonstrative expression skills utilizing a multimodal infant behavior corpus

We have succeeded to obtain the valuable findings about the developmental processes of demonstrative expression skills, which concern the fundamental human .commonsense knowledge, such as to get an object and to catch someone’s attention. We have already developed a framework to record genuine spontaneous speech of infants. We are constructing a multimodal infant behavior corpus, which enables us to elucidate human commonsense knowledge and its acquisition mechanism. Based on the observation utilizing the corpus, we proposed a multimodal behavior description model for the effective observation of demonstrative expressions. We proved that the proposed model has the nearly 90% coverage in an open test of the behavior description task. The analysis results using the model produced many valuable findings from multimodal viewpoints; for example, the change of ‘line of sight’ from ‘object to person’ to ‘person to object’ means that the infant has obtained a better way to catch someone’s attention. Furthermore, the results of intention-based analysis provided us with an infant behavior model which is possible to apply to construct a behavior simulation system.